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•  Usage  of  representation 

-  Transport  vs.  analysis  vs.  storage  vs.  archive 

•  Volume  of  data  informs  representation  choice 

-  Raw  vs.  Summaries 

-  Choice  often  dictates  a  binary  vs.  text  implementation 

•  Policy  Scope 

-  Intra-Organizational 

-  Little  consensus  from  outsiders  necessary 

-  Interoperation  focus 

-  Inter-Organizational 

-  Privacy  issues  more  acute  (sanitization,  filtering) 

-  Common  semantics  are  more  relevant 

-  Efficiency  of  representation  is  more  significant 
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Formats  of  interest 


•  Flow  and  Packet  Formats 

-  IPFIX 

-  PS AMP 

•  Alert  and  Event  Formats 

-  I  DWG 

-  INCH 

•  Context-relevant  Formats 

-  Vulnerability  Report 

-  CRISP 
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Flow  and  Packet  Formats  {de  facto) 
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•  PCAP  (tcpdump) 

-  http://www.tcpdunnp.org 

•  Cisco  NetFlow 
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IETF  IP  Flow  Information  Export  (IPFIX)  WG 


http://www.ietf.org/html.charters/ipfix-charter.html 


•  Binary,  extensible  information  model  for  IP 
flows  exported  from  a  given  observation  point 
(i.e.,  router  line-card)  to  a  coiiector 

-  Based  on  Cisco  Netflow  v9 

•  Designates  a  mandatory  protocol  (SCTP)  to 
use  in  the  transport  of  these  flows 

(Note:  Various  text  and  figures  were  taken  from  the  IPFIX  l-Ds) 
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•  [A]  set  of  IP  packets  passing  an 

observation  point ...  during  a  certain  time 
interval.  All  packets  belonging  to  a  particular 
flow  have  a  set  of  common  properties  [named 
flow  keys].” 

-  One  or  more  packet  header  field  (e.g.  destination  IP 
address),  transport  header  field  (e.g.  destination  port 
number),  or  application  header  field  (e.g.  RTP  header  fields) 

-  One  or  more  characteristics  of  the  packet  itself  (e.g.  number 
of  MPLS  labels) 

-  One  or  more  fields  derived  from  packet  treatment  (e.g.  next 
hop  IP  address,  output  interface) 
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IPFIX  Flow  Definition 
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(2) 


•  A  flow  is  defined  by  a  flow  type  function  that 
considers  the  various  flow  keys 


•  Flexible  definition  provides  support  for: 

-  Filtering 

-  Sampling 

-  Bi-directional  and  unidirectional  flows 
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•  Template-based  format 

-  IPFIX  merely  specifies  the  possible 

-  data  types  (e.g.,  IPv4  address,  octet)  and  the 

-  information  items  (e.g.,  icmpTypeCode,  egress  Interface) 

-  Information  items  are  unique  identifiers  registered 
with  lANA  or  escaped  via  a  vendor  code 

-  A  template  is  merely  an  ordered  list  of  pairs: 
<information  items  (i.e.,  fieldID),  data  length> 

-  No  static  format;  can  be  dynamically  generated  during 
the  export  process 
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IPFIX  Information  Model 

•Two  classes  of  records 

-  Template  Records 

-  Describe  a  format 

-  Data  Records 

-  Contain  data  encoded  and  formatted  according  to  a 
Template  record 
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(2) 


•  Two  flavors  of  Data  Records;  those  that 
encode  the: 

-  Data  stream  (e.g.,  observed  flows),  and 

-  Control  Information  (e.g.,  selection  criteria) 
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IPFIX  Information  Model 

•  4-basic  record  types 

-  Flow  Data  Template 

-  A  description  for  data  record  structure 

-  Flow  Data  Record 

-  IP  flows  formatted  according  to  the  Flow  Data  Template 

-  Option  Template 

-  A  description  of  the  option  record  structure 

-  Option  Record 

-  Control  information  formatted  according  to  the  Option 
Template  Record 
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+ - + - + 

I  I  + - +  + - +  + - +  I 

I  Message  |  |  Data  |  ...  |  Data  I  . . .  I  Data  |  | 

I  Header  |  |  Template |  ...  |  Record  I  . . .  I  Record  |  | 

I  I  + - +  + - +  + - +  I 

+ - + - + 

•  Template  records  are  sent  inline  with  the  data 
records 

-  Frequency  dictated  by  the  quality  of  transport 

-  Possible  to  send  no  template  in  an  export,  and 
reference  a  previously  sent  template  in  the  data 
record 

-  Collector  must  cache  data  templates 
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IPFIX  Message  Header 


0 


1 


2 


3 


01234567890123456789012345678901 

I  Version  Nuinber  |  Length  | 

I  Export  Time  | 

I  Sequence  Number  | 

I  Source  ID  | 


*  128-byte  preamble  sent  with  each  export 
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Src  IP  addr.  |  Dst  IP  addr.  |  Packet  |  Bytes 

I  I  Number  |  Number 


198.168.1.12  I  10.5.12.254  |  5009  |  5344385 

192.168.1.27  I  10.5.12.23  |  748  |  388934 


Flow 
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IPFIX  Example:  Template 


(2) 


0 


1 


2 


3 


01234567890123456789012345678901 
I  FlowSet  ID  =  0  I  Length  =24  bytes  | 
I  Template  ID  256  |  Field  Count  =  4  | 
I  IP_SRC_ADDR  =  0x0008  |  Field  Length  =  4  | 
I  IP_DST_ADDR  =  0x0 OOC  |  Field  Length  =  4  | 
I  IN_PKTS  =  0x0002  |  Field  Length  =  4  | 
I  IN_BYTES  =  0x0001  |  Field  Length  =  4  | 
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IPFIX  Example:  Data 


0  12 
0123456789012345678901234567 

I  FlowSet  ID  =  256  |  Length  =  36 

I  198.168.1.12 

I  10.5.12.254 

I  5009 

I  5344385 

I  192.168.1.27 

I  10.5.12.23 

I  748 

I  388934 
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(3) 

3 

9  0  1 
+-+-+-+ 

I 

+-+-+-+ 

I  #1 

+-+-+-+ 

I 

+-+-+-+ 

I 

+-+-+-+ 

I 

+-+-+-+ 

I  #2 

+-+-+-+ 

I 

+-+-+-+ 

I 

+-+-+-+ 

I 

+-+-+-+ 
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•  Reliable  service 

-  TCP  equivalent 

•  “Partially  reliable”  service 

-  During  un-congested  periods,  all  the  records 
marked  for  deletion  under  congestion  will  be 
reliably  delivered 

-  During  congested  periods,  the  exporter  will  drop 
packets  to  protect  the  network 
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•  Requirements  for  IP  Flow  Information  Export 

-  draft-ietf-ipfix-reqs-16 

•  Architecture  Model  for  IP  Flow  Information 
Export 

-  draft-ietf-ipfix-architecture-03 

•  Information  Model  for  IP  Flow  Information 
Export 

-  draft-ietf-ipfix-info-03 

•  IPFIX  Protocol  Specifications 

-  draft-ietf-ipfix-protocol-03 
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IETF  Packet  Sampling  (PSAMP)  WG 


http://www.ietf.org/html.charters/psamp-charter.html 


•  Binary,  extensible  information  model  for 
specifying 

-  Selection  operations  (sampling  and  filtering)  on  a 
packet  stream,  and 

-  Packets  yielded  by  the  selection  operation 

•  Designates  a  mandatory  protocol  (IPFIX)  to 
use  in  the  transport  of  these  packets 
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Relationship  between  IPFIX  and  PSAMP 


•  PSAMP  extends  the  IPFIX  data  model 

-  A  PSAMP  data  record  is  an  special  instance  of  an 
IPFIX  flow  record  with  different  semantics 

-  i.e.,  a  flow  record  with  only  a  single  packet 

-  Augments  the  IPFIX  data  model  to  support 
Selection  Process 


•  PSAMP  reuses  the  IPFIX  transport  protocol 
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•  Sampling 

-  “Provisioning  of  information  about  a  specific 
characteristic  of  the  parent  population  at  a  lower 
cost  than  a  full  census  would  demand” 

•  Filtering 

-  Deterministic  selection  of  packets  based  on  the 

-  packet  content 

-  treatment  of  the  packet  at  the  observation  point,  or 

-  functions  operating  on  the  selection  state. 

•  Possible  to  create  schemes  combing  of  both 
sampling  and  filtering  selections 
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•  Systematic  Sampling  (deterministic  function) 

-  Count-based  (spatial  packet  position;  e.g.,  packet 
count) 

-  Time-based  (temporal  packet  position;  e.g.,  arrival 
time) 

•  Random  Sampling 

-  n-out-of-N 

-  Probabilistic 

-  Uniform  Probabilistic  (same  probability  for  each  packet) 

-  Non-Uniform  Probabilistic  (probability  depends  on  input) 

-  Flow  State  Probabilistic 

-  Sampling  probability  depends  on  flow  state 
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•  Match/Mask 

-  Apply  bit  mask  to  the  header  or  the  first  N-bytes 

•  Hashing 

-  Apply  a  hash  function  to  the  header  or  first  N-byte 

•  Packet  Features 

-  Properties  of  the  packet  header 

•  Router-state  selection 

-  Properties  of  the  route  or  packet  treatment 
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•  A  Framework  for  Passive  Packet 
Measurement 

-  draft-ietf-psamp-framework-05 

•  Sampling  and  Filtering  Techniques  for  IP 
Packet  Selection 

-  draft-ietf-psannp-sannple-tech-04 

•  Packet  Sampling  (PSAMP)  Protocol 
Specifications 

-  draft-ietf-psamp-protocol-OI 

•  Information  Model  for  Packet  Sampling 
Exports 

-  draft-ietf-psamp-info-OI 
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IETF  Intrusion  Detection  WG  (IDWG) 


http://www.ietf.org/html.charters/idwg-charter.html 


•  XML  information  model  for  network  and  host- 
based  Intrusion  Detection  System  alerts 

-  Intrusion  Detection  Message  Exchange  Format 
(IDMEF) 

•  Defines  a  protocol  to  exchange  these  alerts 

-  Intrusion  Detection  Exchange  Protocol  (IDXP) 

-  BEEP-based  profile  to  exchange  IDMEF 
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Sensor  properties 
Timestamps 

Source/Target  characteristics 
•  IP  address,  ports 
Impact  assessment 
Event  classification 
Extension  mechanism 
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•  Intrusion  Detection  Message  Exchange 
Requirements 

-  draft-ietf-idwg-requirements-1 0 

•  The  Intrusion  Detection  Message  Exchange 
Format 

-  draft-ietf-idwg-idmef-xml-12 

•  The  Intrusion  Detection  Exchange  Protocol 
(IDXP) 

-  draft-ietf-idwg-beep-idxp-07 

•  The  TUNNEL  Profile 

-  Rfc3620 
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http://www.ietf.org/html.charters/inch-charter.html 


•  XML  information  model  for  exchanging 
“incident  data”  among  CSIRTs 

-  Incident  Object  Description  Exchange  Format 
(iODEF) 

•  No  exchange  protocol  specified 
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•  Extensible  framework  to  exchange  information 
between  CSIRTs 

-  Workflow 

-  incident  identifiers,  conveying  expectations,  data  usage 
restrictions 

-  Incident  description  and  conclusions 

-  Source/Destination  information 

-  Contact  information 

-  References  to  vulnerabilities,  advisories,  and  artifacts 

-  Classification  and  impact  assessments 

•  Extensions 

-  RID:  DoS  traceback  for  ISPs 

-  (possible)  Anti-Spam  lists 
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•  Requirements  for  Format  for  INcident  Report 
Exchange  (FINE) 

-  draft-ietf-inch-requirements-03 

•  The  Incident  Data  Exchange  Format  Data 
Model 

-  draft-ietf-inch-iodef-02 

•  The  Incident  Object  Description  Exchange 
Format  (lODEF)  Implementation  Guide 

-  draft-ietf-inch-implement-00 

•  Real-Time  Inter-Network  Defense 

-  draft-ietf-inch-rid-00 
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http://www.ietf.org/html.charters/crisp-charter.html 


•  XML,  extensible  information  model  for  global 
registry  information 

-  i.e.,  Whois  with  structure 

•  Designates  a  mandatory  protocol  (BEEP)  for 
the  query/response  exchange 
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•  Mitre  CVE 

-  http://cve.nnitre.org/ 

•  Mitre  OVAL 

-  http://oval.nnitre.org/ 

•  NISTiCAT 

-  http://icat.nist.gov/icat.cfm 
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•  Common  Advisory  Interchange  Format 
(CAIF) 

-  RUS-CERT 

-  http://cei1.uni-stuttgai1.de/projects/caif/ 

•  Advisory  and  Notification  Markup  Language 
(AN  ML) 

-  OpenSec 

-  http://www.opensec.org/anml/ 

•  Application  Vulnerability  Description 
Language  (AVDL) 

-  OASIS 

-  http://www.oasis- 

open.org/committees/tc_home. php?wg_abbrev=avdl 
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Relevance  of  the  Formats  to  Flows 

•  IPFIX 

-  storage  and  transport  format  for  flows 

•  PS  AMP 

-  Describe  acquisition  process  of  the  flows 

•  IDMEF 

-  Describe  events  created  from  flows 

•  lODEF  (with/without  extensions) 

-  Describe  flow  summaries,  baselines,  etc. 
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•  Packets  and  Flow  Formats 

-  IPFIX:  implementations  exist  (e.g.,  Argus) 

-  PSAMP:  work  in  progress 

•  Alerts  and  Events  Formats 

-  IDMEF:  adoption  only  in  Snort,  Prelude,  Arcsight 

-  lODEF:  adoption  by  5-15  CSIRTs  in  Europe,  Asia, 
and  the  US 

•  Context  Formats 

-  Vulnerability  formats:  work  in  progress,  some  used 
in  closed  communities 

-  CRISP:  work  in  progress 
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